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ABSTRACT 


Communications  satellite  system  architecture  trades  traditionally  only  consider  the  cost 
per  unit  of  capacity  provided.  This  selection  method  ignores  the  other  requirements  with 
which  the  system  architectures  were  designed,  and  that  are  critical  to  providing  a 
capability  to  the  warfighter.  A  survey  of  communications  satellite  systems  identified  five 
common  attributes  that  are  incorporated  in  the  design  process:  communications  capacity, 
access,  interoperability,  commandability,  and  infonnation  assurance  and  protection.  A 
mathematical  model  was  implemented  to  enable  the  analysis  of  communications  satellite 
system  architectures  based  on  multiple  system  attributes.  Utilization  of  the  model  in  a 
hypothetical  test  case  demonstrated  how  variations  in  key  perfonnance  attributes 
influences  the  choice  of  the  preferred  system  in  a  selection  process. 
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EXECUTIVE  SUMMARY 


Communications  satellite  system  architecture  trades  traditionally  consider  only  the  cost 
per  unit  of  capacity  provided.  But  a  successful  architecture  has  many  critical  attributes 
that  meet  documented  and  validated  requirements.  Making  a  decision  based  solely  on 
cost  per  unit  of  capacity  provided  ignores  critical  requirements  essential  to  providing  a 
capability  to  the  warfighter.  To  ensure  these  requirements  are  considered  during  trades,  a 
method  to  account  for  additional  system  attributes  in  the  execution  of  architecture  trades 
was  implemented. 

A  survey  of  communications  satellites  systems  identified  five  common  attributes 
that  are  incorporated  in  the  system  design  process.  These  are  communications  capacity, 
access,  interoperability,  commandability,  and  information  assurance  and  protection. 
Additional  system  specific  attributes  can  be  based  on  Technical  Performance  Measures 
(TPMs)  and  Key  Performance  Parameters  (KPPs)  for  the  actual  system  solutions  to  be 
analyzed. 

A  mathematical  model  was  implemented  to  enable  the  analysis  of 
communications  satellite  system  architectures  based  on  the  perfonnance  of  multiple 
system  attributes.  The  mathematical  model  supports  any  number  of  attributes  desired  for 
analysis.  All  selected  attributes  are  allowed  to  have  a  different  relative  importance  based 
on  the  preference  of  the  stakeholders.  Utilization  of  the  model  in  a  hypothetical  case 
indicated  that  a  system  selection  considering  additional  key  performance  attributes  can  be 
different  from  the  traditional  solution  of  narrowly  focusing  on  one  parameter. 
Understanding  the  reasoning  behind  the  difference  will  help  ensure  the  best  system  is 
selected. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Satellite-based  communications  provide  a  crucial  capability  to  the  U.S.  military  in 
the  execution  of  their  missions.  The  need  for  a  worldwide  exchange  of  voice,  video,  and 
data  during  a  military  operation  has  resulted  in  the  development  and  operation  of  satellite 
systems  that  provide  a  global  communication  capability.  The  operational  benefits 
afforded  by  satellite-based  communications  were  recognized  from  the  beginning  of  the 
U.S.  space  program  and  continues  to  remain  a  fundamental  mission  area  (Martin,  2001). 

Satellite  communications  enable  a  level  of  pervasive  coverage  that  is  not  possible 
by  any  other  means.  Satellite  communications  also  provide  coverage  in  remote  areas  that 
lack  infrastructure,  such  as  at  sea,  or  in  developed  areas  where  communications 
infrastructure  has  been  incapacitated  by  natural  disasters,  war,  or  other  extreme 
circumstances.  Additionally,  demand  for  communications  throughput  continues  to  grow 
as  more  data  is  exchanged  between  operators  and  sensors  in  a  theater  of  engagement  and 
remotely  located  analysts,  planners,  and  commanders. 

In  order  to  efficiently  meet  communications  throughput  demand,  potential  system 
architecture  solutions  are  developed  and  then  analyzed  in  trade  studies.  As  the  system 
designs  are  generated  all  requirements  are  included  and  balanced  to  meet  specified  needs. 
Traditionally  the  selection  of  a  communications  satellites  systems  architecture  solution  is 
based  upon  the  evaluation  of  a  single  criterion:  the  cost  per  unit  of  system  communication 
capacity,  or  throughput.  This  approach  comes  from  the  requirement  to  be  compliant  with 
the  DoD  5000  Series  Policy  regarding  cost  analysis  coupled  with  a  comparison  to 
commercial  options  (DoD  Directive  5000.01,  2007  and  DoD  Instruction  5000.02,  2008], 

Communications  satellite  system  architecture  selection  based  solely  on  the  cost 
per  unit  of  system  communication  capacity  clearly  omits  documented  and  validated 
requirements  from  the  decision  making  process.  Examples  of  these  requirements  may 
include  architecture  robustness,  the  ability  of  the  system  design  to  operate  in  a  variety  of 
situations;  and  interoperability  with  other  existing  or  planned  equipment,  tactics, 
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procedures,  and  systems.  Thus,  a  method  to  assess  communications  satellite  system 
architectures  based  on  multiple  attributes  simultaneously  is  desired. 

B.  PURPOSE 

The  purpose  of  this  thesis  was  to  implement  a  mathematical  model  for  assessing 
communications  satellite  system  architectures  based  on  the  satisfaction  of  multiple 
perfonnance  attributes. 

C.  RESEARCH  QUESTIONS 

The  following  three  questions  are  addressed. 

1.  What  are  the  key  quantifiable  architectural  attributes  that  contribute  to 
meeting  the  users’  requirements  of  a  communications  satellite? 

2.  What  is  an  appropriate  mathematical  model  for  evaluating  communications 
satellite  architectures? 

3.  How  can  such  a  mathematical  model  be  applied  to  the  assessment  of  a 
communications  satellite  architecture? 

D.  SCOPE  AND  METHODOLOGY 

The  thesis  proposes  a  methodology  that  can  be  utilized  to  perfonn  system 
architecture  trades  on  DoD  satellite  communications  architectures  that  require  multiple 
satellites  to  meet  global  communications  needs.  First,  key  communications  satellite 
system  architecture  attributes  were  indentified.  Next,  a  mathematical  model  to  compare 
system  architecture  solutions  was  presented.  Finally,  the  mathematical  model  was  applied 
to  evaluate  a  set  of  hypothetical  communications  satellite  system  architectures. 

E.  BENEFITS  OF  THIS  STUDY 

The  outcome  of  this  research  could  be  used  to  support  communications  satellite 
system  architecture  trades.  The  application  of  this  model  may  allow  decision  makers  to 
be  more  infonned  during  system  architecture  selection  than  if  relying  on  the  traditional 
approach  that  considers  only  system  communications  capacity  and  cost. 
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II.  MODEL  SELECTION 


A.  INTRODUCTION 

This  chapter  discusses  the  mathematical  model  to  be  utilized  to  conduct  trade 
studies  on  candidate  communications  satellite  architectures.  First,  a  survey  of  candidate 
system  parameters  upon  which  to  base  the  mathematical  model  was  conducted.  Next, 
potential  mathematical  models  were  considered.  Finally,  a  mathematic  model  was 
presented  to  assess  the  perfonnance  of  the  communications  satellite  system  architectures. 

B.  IDENTIFICATION  OF  KEY  SYSTEM  ARCHITECTURE  ATTRIBUTES 

1.  Survey  of  Common  Technical  Performance  Measures,  Key 
Performance  Parameters,  and  System  Requirements 

A  survey  of  open  source  literature  identified  key  architecture  attributes  of  military 
satellite  communications  systems.  This  set  of  attributes  is  not  specific  to  an  existing  or 
planned  satellite  program,  but  is  intended  to  be  representative  of  multiple  system  design 
options.  Some  of  the  attributes  may  be  applicable  to  non-military  satellite  communication 
systems,  and  should  be  highlighted  when  considering  commercial  solutions  that  support 
the  current  national  space  policy  to  “pursue  potential  opportunities  for  transferring 
routine,  operational  space  functions  to  the  commercial  space  sector  where  beneficial  and 
cost-effective,  except  where  the  government  has  legal,  security,  or  safety  needs  that 
would  preclude  commercialization”  (Office  of  the  President  of  the  United  States,  2010,  p. 
14). 

The  five  key  attributes  are:  communications  capacity,  access,  interoperability, 
commandability,  and  information  assurance  and  protection.  The  descriptions  of  these 
attributes  follow. 


a.  Communications  Capacity 

Capacity  is  the  most  fundamental  measure  for  a  communications  system 
and  is  common  to  both  commercial  and  military  satellite  systems.  This  attribute  measures 
the  amount  of  data  communicated  per  unit  of  time  (Gigabits  per  second  (Gbps)  for 
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example).  Alternatively,  this  parameter  can  also  be  expressed  as  the  available  frequency 
bandwidth  per  unit  time.  The  frequency  metric  may  be  more  useful  if  it  is  desirable  to 
normalize  a  performance  measure  that  is  independent  of  signal  modulation  schemes.  For 
the  purposes  of  the  mathematical  model  presented  in  this  thesis,  the  choices  of  units  for 
capacity  are  left  to  the  expert  analyzing  the  system.  The  only  requirement  is  that  the 
method  utilized  to  predict  capacity  is  applied  in  the  same  manner  to  all  system  solutions 
being  considered  (Kakavas,  Ha,  &  Garcia,  1998,  GAO  94-48). 

b.  Access 

Access  is  the  ability  of  the  system  to  provide  communication  in  a  defined 
location  when  needed.  The  coverage  area  defines  the  global  locations  in  which 
communications  are  expected  to  be  provided.  Commercial  satellite  communications 
systems  have  coverage  areas  generally  limited  to  population  centers  to  ensure  sufficient 
revenue  from  operating  the  systems.  Population  centers  are  quite  stable,  essentially  fixed 
over  5-15  years  of  a  satellite’s  operational  life.  Military  needs  require  communications 
capability  in  remote  areas,  be  it  sparsely  inhabited  land  areas  or  open  ocean.  Additionally, 
the  areas  to  be  covered  often  change,  whether  simply  from  the  movement  of  units  or  from 
new  areas  of  operation;  these  new  operational  areas  can  arise  from  training  operations, 
disaster  relief,  and  armed  conflict.  The  “when  needed”  part  of  access  is  not  only  the 
expected  operational  lifetime  of  the  system,  but  also  incorporates  the  need  for  a  satellite 
system  architecture  to  provide  communications  coverage  in  both  nominal  and  stressed 
conditions.  A  nominal  operating  condition  would  include  a  complete  satellite 
constellation  supporting  missions  with  concentrations  and  locations  of  users  in 
accordance  with  system  design  scenarios.  A  stressed  condition  could  be  the  loss  of  a 
satellite  due  to  circumstances  such  as  system  failure,  the  inability  to  secure  a  commercial 
lease,  or  a  distribution  of  users  that  was  not  envisioned  in  the  system  design  process 
(Bradley  1997,  Kakavas,  Ha,  &  Garcia,  1998,  GAO/NSIAD-93-216,  GAO  94-48). 

c.  Interoperability 

Interoperability  is  a  capability  requirement  for  DoD  systems.  For  satellite 
communications  architectures  this  includes  the  system’s  ability  to  work  with  both 
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existing  and  planned  terminals.  The  set  of  tenninals  supported  is  likely  to  include  all  U.S. 
military  services  and  allied  systems  used  in  a  joint  international  effort.  Quantifying  the 
number  of  specified  user  terminals  that  a  system  architecture  can  support  is  one  method 
to  measure  interoperability.  Other  interoperability  metrics  may  also  be  defined  specific  to 
a  system  requirements  set  (Wickline  1998,  pp.  5-10). 

d.  Com  m  andability 

Commandability  refers  to  the  operator’s  ability  to  command  the  satellite 
per  the  system  requirements.  Commandability  may  be  comprised  of  a  number  of  factors 
including  the  level  of  protection  of  ground  operations  centers  against  threats  both 
manmade  and  natural,  the  time  necessary  to  switch  between  system  configurations, 
autonomous  operations,  and  system  level  planning  functions  (Bradley  1997,  GAO  94- 
48). The  ability  of  the  system  to  comply  with  these  requirements  defines  the  perfonnance 
for  this  attribute. 


e.  Information  Assurance  and  Protection 

The  policy,  requirements,  and  implementation  of  infonnation  assurance 
and  information  protection  will  be  defined  in  the  satellite  system  requirements 
documentation  (DoD  Instruction  5000.02,  2008).  The  ability  of  the  system  to  comply 
with  these  requirements  defines  the  perfonnance  for  this  attribute.  The  policies  and 
regulations  continue  to  evolve  with  the  Defense  Infonnation  Systems  Agency  (DISA)  as 
the  lead  for  the  DoD.  As  system  architectures  are  traded,  it  is  important  to  consider  not 
only  compliance  of  the  system  at  the  time  that  the  trades  are  executed,  but  also  the  ability 
of  the  system  to  evolve  and  adapt  over  its  lifetime. 

2.  Selection  of  Key  Quantifiable  System  Architecture  Attributes 

Since  the  purpose  of  the  model  is  to  consider  attributes  that  contribute  to  meeting 
vetted  requirements,  these  five  attributes  appear  to  be  at  the  core  of  communications 
systems  architectures  and  should  be  included  in  the  model  as  applicable.  Thus,  it  is  not 
necessary  to  use  all  five  of  the  representative  attributes  in  the  mathematical  model,  but 
only  those  with  significant  requirements  relevance.  Of  the  five  key  attributes  above,  only 
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the  communications  capacity  should  be  deemed  required  in  the  mathematical  model  to 
maintain  traceability  to  the  traditional  modeling  approach. 

Beyond  these  five  key  attributes,  there  may  be  other  requirements  that  are  specific 
the  communications  satellite  system  that  lead  to  the  inclusion  of  additional  attributes  in 
the  mathematical  model.  Key  Perfonnance  Parameters  (KPPs),  Technical  Perfonnance 
Measures  (TPMs),  and  other  system  requirements  should  be  assessed  for  relevance  and 
utilized  in  the  model  as  applicable.  KPPs  are  a  minimum  set  of  perfonnance  parameters 
that  guide  a  system  development  effort.  KPPs  are  identified  by  users,  approved  by  the 
requirements  authority,  and  contained  in  the  system  Capability  Description  Document 
(CDD)  (DoD  Instruction  5000.02,  2008).  TPMs  are  quantitative  values  that  describe 
system  performance  (planned,  predicted,  or  measured),  and  are  often  related  to  KPPs 
(Blanchard  &  Fabrycky,  2006,  pp.  75-78;  The  Defense  Acquisition  Guidebook, 
http://at.dod.mil/docs/DefenseAcquisitionGuidebook.pdf,  TPM  definition  and  use). 

The  following  three  considerations  should  be  employed  in  the  selection  of  the  key 
system  architecture  attributes  for  incorporation  into  the  mathematical  model. 

First,  a  key  attribute  used  in  the  mathematical  model  must  be  quantifiable  with 
distinctly  defined  threshold,  or  minimum  acceptable,  and  objective,  or  desired,  levels. 
Qualitative  assessments  can  be  translated  to  numeric  values;  however,  qualitative 
assessments  should  be  avoided  when  possible.  If  the  threshold  and  objective  values  are 
the  same,  or  if  all  satellite  system  architectures  provide  the  same  level  of  perfonnance, 
then  the  attribute  will  not  be  useful  for  distinguishing  between  architecture  solutions,  and 
therefore  should  not  be  included  in  the  analysis. 

Second,  attributes  that  have  a  range  of  levels  achievable  between  threshold  and 
objective  levels  should  be  utilized  in  the  mathematical  model  whenever  possible.  The 
range  of  capability  will  enable  the  mathematical  model  to  more  readily  rank  the  various 
architectures.  Extensive  use  of  attributes  that  only  meet  or  fail  to  meet  objective 
performance  may  result  in  grouping  of  architectures  into  two  categories.  Attributes  that 
do  not  have  a  range  of  levels  achievable  between  the  threshold  and  objective  levels  are 
discouraged  but  not  precluded  because  there  may  be  situations  in  which  this  ability  to 
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provide  objective  performance  is  highly  desirable.  Examples  may  include  an  ability  to 
survive  a  threat,  or  compatibility  with  other  existing  systems. 

Third,  attributes  should  be  unique  and  not  derived  from  each  other.  This  can  be 
done  by  avoiding  the  utilization  of  both  primary  and  derived  requirements.  An  example 
of  a  poor  choice  could  be  selecting  both  the  Equivalent  Isotropically  Radiated  Power 
(EIRP)  and  link  data  rate.  Because  the  data  rate  is  a  function  of  the  available  power 
(Pritchard,  Suyderhoud,  &  Nelson,  1993),  they  are  interdependent.  Remaining  focused  on 
high  level  system  attributes  avoids  including  dependent  relationships  in  the  model. 

C.  SELECTION  OF  A  MATHEMATICAL  MODEL  TO  ASSESS  SYSTEM 

ARCHITECTURES 

The  mathematical  model  selected  to  assess  system  architectures  is  intended  to 
replace  the  existing  approach  of  calculating  a  cost  per  unit  capacity.  In  order  to  have  the 
new  model  accepted  as  a  replacement  it  must  incorporate  the  following  elements.  First, 
the  model  must  be  able  to  calculate  a  cost  per  capability  provided  to  be  compliant  with 
DoD  5000  that  requires  system  trades  to  be  conducted  in  consideration  of  cost.  Second, 
the  model  must  be  understandable  by  decision  makers  who  use  the  data  from  the  current 
cost  per  capacity  method.  To  this  end,  it  is  desirable  to  have  a  method  that  can  replicate 
the  results  of  the  traditional  model  for  comparison.  It  is  also  necessary  for  the  model 
approach  and  results  to  be  clearly  explainable  to  stakeholders  or  else  the  model  will  not 
be  used. 

1.  Summary  of  Potential  Mathematical  Model  Forms 

Once  a  selection  of  key  attributes  is  made  it  is  necessary  to  have  a  methodology  to 
implement  for  the  assessment  of  the  system  architectures.  A  mathematical  model 
incorporates  multiple  attributes  and  avoids  subjectivity  by  quantifying  system 
performance.  It  is  expected  that  attributes  used  in  the  assessment  of  the  architectures  will 
have  varying  levels  of  importance.  When  system  architecture  trades  are  conducted  on  the 
system  capacity  attribute  alone,  the  decision  making  process  does  not  benefit  from  the 
careful  consideration  of  other  requirements-driven  system  architecture  attributes. 
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One  mathematical  model  that  can  be  utilized  in  this  analysis  is  termed  the 
Weighting  Method.  In  this  approach,  the  attributes  are  first  assigned  a  weight,  with  the 
sum  of  all  the  weights  equal  to  one.  A  performance  rating  is  then  calculated  for  each 
attribute  and  the  product  of  the  weight  and  performance  rating  is  computed.  Finally,  the 
sum  of  all  weight  and  perfonnance  products  for  each  communication  satellite  system 
architecture  alternative  is  calculated.  The  resultant  sum,  or  score,  can  be  utilized  to 
compare  the  scores  from  other  communication  satellite  system  architectures.  The  system 
architecture  with  the  highest  score  is  the  most  preferred  (Blanchard  &  Fabrycky,  2006, 
pp.  178-180). 

In  a  specific  application  of  the  Weighting  Method,  the  attribute  weights  are  given 
by  the  stakeholders.  Next,  the  perfonnance  scores  are  assessed  for  each  attribute  and 
system  based  on  the  achieved  performance  normalized  to  the  range  defined  by  the 
threshold  and  objective  levels.  Specifically,  the  score  achieved  is  a  value  between  0  and 
1,  where  0  represents  achieving  only  the  threshold  level  and  1  represents  achieving  the 
objective  level.  If  a  system  under  consideration  does  not  meet  the  threshold  performance 
value  for  an  attribute,  then  a  new  threshold  value  must  be  specified  and  the  model  must 
be  updated  with  the  new  value.  Only  when  all  systems  meet  the  threshold  for  every  key 
attribute  can  we  ensure  all  systems  are  being  assessed  equally.  The  highest  perfonnance 
score  is  1.0.  No  additional  benefit  is  assessed  for  a  system  that  exceeds  the  objective 
performance  level.  An  overall  score  for  each  system  architecture  is  calculated  using  the 
sum  the  products  of  the  weight  and  score  for  each  attribute.  This  value  is  called  the 
Overall  Measure  Of  Effectiveness  (OMOE).  The  system  selection  is  based  on  the  highest 
OMOE  (Laverghetta,  1998;  Hootman  &  Whitcomb,  2005).  This  approach  will  be  referred 
to  as  the  OMOE  Method. 

Another  mathematical  modeling  approach  is  the  Analytical  Hierarchy  Process 
(AHP)  developed  by  Thomas  Saaty  in  the  1970s  (Saaty,  1980).  This  process  is  another 
application  of  the  Weighting  Method,  with  the  focus  on  the  computation  of  a 
performance  rating.  Calculation  of  the  perfonnance  rating  is  accomplished  by  comparing 
the  perfonnance  between  every  system  architecture  pair  for  each  performance  attribute  to 
determine  which  architecture  is  prefened  and  how  strongly.  The  assessment  of  preference 
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can  be  done  by  group  or  individually.  The  AHP  process  is  subjective  and  can  be  time 
consuming  since  the  number  of  assessments  grows  geometrically  as  the  product  of  the 
number  of  systems  assessed  and  the  number  of  key  attributes  selected  (Ragsdale,  2008, 
pp.  777-784;  Kaymaz  &  Diri,  2008,  Tsagdis,  2008). 

Value  Engineering  is  a  technique  that  can  be  utilized  to  assess  projects  and 
identify  opportunities  for  cost  savings  and  cost  avoidance  while  optimizing  perfonnance 
and  productivity  in  a  wide  variety  of  applications  including  hardware,  software,  and 
infrastructure  projects  (Mandelbaum  &  Reed,  2006).  Value  Engineering  is  attributed  to 
General  Electric  around  1948  when  a  process  initially  named  Value  Analysis  was 
developed  and  applied  to  control  production  costs  (Miles  &  Reger,  1958).  This  process 
has  been  updated  over  time  and  included  in  the  United  States  Office  of  Management  and 
Budget  Circular  A131  (Office  of  Management  and  Budget,  1993).  The  purpose  of  the 
Value  Engineering  technique  is  to  provide  best  value  system  solution  to  both  the 
customer  and  producer.  The  key  principle  of  the  Value  Engineering  technique  is  to 
consider  all  decisions  and  opportunities  in  light  of  the  total  system  cost.  The  total  system 
cost  includes  not  only  the  cost  to  acquire  the  system  that  include  development, 
manufacturing,  and  materials,  but  also  costs  to  field,  operate,  maintain,  upgrade,  and 
dispose  of  the  system.  This  approach  can  be  used  to  account  for  the  total  cost  of  obtaining 
a  level  of  perfonnance  that  exceeds  the  threshold  requirement  as  not  every  increase  in 
capability  is  worth  the  cost  to  achieve  the  benefit. 

2.  Selection  of  Mathematical  Model 

The  traditional  system  architecture  selection  process  is  a  comparison  of  the  cost 
per  unit  of  system  communication  capacity.  While  this  approach  provides  results  that  are 
easy  to  understand,  it  can  also  result  in  an  architecture  choice  that  ignores  variation  in 
other  significant  system  requirements.  Examples  of  key  requirements  that  are  ignored  by 
this  approach  include,  but  are  not  limited  to:  access;  interoperability;  commandability; 
and  infonnation  assurance  and  protection.  In  the  traditional  approach  there  is  no  way  to 
consider  tradeoffs  between  capacity  and  the  other  key  system  requirements.  This  could 
result  in  the  preference  and  selection  of  an  architecture  that  has  significantly  more 
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capacity  in  selected  geographic  areas  over  one  that  provides  more  uniform  global 
coverage  capability.  Another  example  could  be  the  selection  of  a  system  architecture  that 
provides  marginally  more  capacity  over  another  that  has  more  capability  for  information 
assurance  and  protection. 

A  mathematical  model  that  combines  the  principles  of  the  Value  Engineering 
technique  and  OMOE  Method  has  been  selected  for  use.  Specifically,  calculating  the  total 
system  cost  per  OMOE  per  number  of  years  at  Full  Operational  Capability  (FOC)  for 
each  system  architecture.  Including  the  number  of  years  at  FOC  is  critical  for  comparing 
system  architecture  solutions  that  may  operate  for  different  periods  of  time.  This 
mathematical  model  meets  the  criteria  presented  in  II.A.2  above.  Use  of  Value 
Engineering  technique  establishes  that  the  total  system  cost  be  utilized  in  the  system 
architecture  trades.  The  OMOE  Method  can  be  simplified  to  consider  only  the  system 
capacity,  or  be  expanded  to  incorporate  any  number  of  key  attributes.  The  overall  method 
that  defines  a  cost  per  OMOE  results  in  an  understandable  decision  making  approach 
where  the  lowest  cost  per  OMOE  solution  is  preferred. 

The  AHP  method  was  not  selected  due  to  its  complexity  and  inherent  difficulty  in 
tracing  results  back  to  cost  per  unit  of  capacity  (Ragsdale,  2008,  pp.  777-784;  Kaymaz  & 
Diri,  2008;  Tsagdis,  2008).  The  AHP  method  could  be  still  be  utilized  to  detennine  the 
weighting  factors  of  the  attribute;  however,  that  choice  was  omitted  in  this  assessment 
approach. 


a.  Model  Limitation 

There  is  a  limitation  with  the  selected  approach  in  the  rare  instance  that 
one  of  the  system  architectures  only  provides  the  threshold  perfonnance  levels  for  all  key 
attributes.  In  this  instance  the  OMOE  would  be  zero,  resulting  in  a  divide  by  zero  error  in 
the  computation  of  the  final  cost  per  capability  calculation.  To  overcome  this  issue  it  is 
necessary  to  adjust  the  OMOE  of  the  threshold  only  system  to  make  it  greater  than  zero. 

It  is  suggested  that  initially  the  OMOE  for  the  threshold-only  system  be 
set  to  half  that  of  the  next  higher  OMOE.  Further  investigation  of  the  other  system 
architectures  will  need  to  be  conducted  to  determine  if  this  is  an  appropriate  adjustment. 
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For  example,  if  the  system  with  the  next  higher  OMOE  has  only  one  of  10  key  attributes 
that  exceeds  threshold  performance  by  a  small  amount,  then  setting  the  threshold-only 
OMOE  to  half  is  appropriate.  However,  if  the  next  higher  OMOE  comes  from  a  system 
that  has  one  or  more  of  10  key  attributes  that  exceeds  threshold  perfonnance  by  a 
significant  amount  then  the  OMOE  for  the  threshold  only  system  architecture  should  be 
less  than  half  A  sensitivity  analysis  of  the  system  trade  off  results  will  be  necessary  to 
assess  the  effects  of  the  OMOE  adjustment  to  the  threshold  only  system. 

3.  Description  of  the  mathematical  model 

Now  that  a  mathematical  model  has  been  selected,  the  model  will  be  described  in 
greater  detail.  The  discussion  will  first  cover  the  inputs,  followed  by  the  output,  and 
finally  the  mathematical  operations  that  transforms  the  inputs  to  the  output. 

a.  Model  Inputs 

The  inputs  to  the  mathematical  model  include  the  following  six 

components: 

(1)  The  key  attributes  used  in  the  detennining  of  the  OMOE  score. 
Quantifiable  values  for  each  attribute  must  be  available  for  each  system  architecture 
being  traded. 

(2)  Threshold  and  objective  perfonnance  levels  for  each  attribute  utilized. 
These  requirements  must  be  the  same  for  every  system  analyzed  by  the  model  to  avoid 
biasing  results  that  would  lead  to  unfair  comparisons.  All  systems  must  also  provide  at 
least  threshold  performance  capability  for  every  key  attribute.  A  conditional  check  of  the 
inputs  with  the  modeling  tool  can  be  used  to  identify  if  there  are  any  violations  of  the 
threshold.  If  any  system  architecture  requires  a  waiver  to  a  threshold  performance  level  a 
waiver  request  will  be  processed  through  the  program’s  fonnal  system  engineering 
process.  If  the  waiver  is  accepted  it  must  be  applied  to  all  communication  satellite 
architectures  and  the  new  threshold  value  must  be  incorporated  in  the  model.  If  the 
waiver  is  not  accepted  then  the  system  or  systems  that  do  not  meet  the  threshold  level 
cannot  be  assessed  using  this  method. 
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(3)  Weighting  factors  for  each  attribute.  The  weights  are  detennined  by  the 
stakeholders.  The  only  requirement  is  the  sum  of  the  attribute  weights  must  be  equal  to 
one. 

(4)  Performance  levels  achieved  for  each  attribute  of  each  system.  When 
recording  the  perfonnance  levels  it  is  important  to  be  sure  that  units  are  consistent  with 
those  used  for  the  threshold  and  objective  performance  levels.  Mixing  units  will  result  in 
erroneous  results. 

(5)  Total  Cost  for  each  communications  satellite  system  architecture  being 
analyzed.  The  methodology  for  computing  total  costs  must  be  consistent  across  all 
systems.  One  might  consider  utilizing  the  time  value  of  money  to  normalize  varying 
funding  profiles  over  multiple  years.  Utilizing  the  life  cycle  cost  of  the  systems  is 
encouraged  as  long  as  the  computation  across  all  systems  is  equivalent.  For  example, 
when  comparing  the  expansion  of  an  existing  system  to  the  development  of  a  new  system 
it  may  be  necessary  to  exclude  from  the  life  cycle  calculation  funds  that  have  previously 
been  expended,  or  that  were  expended  by  other  groups,  such  as  venture  capital. 

(6)  Time,  in  years,  at  full  operational  capability  (FOC).  This  measure  is  necessary 
to  nonnalize  across  potentially  different  time  periods  of  FOC.  The  model  will  neglect  to 
provide  a  value  for  any  performance  provided  as  the  system  architecture  ramps  up  to 
FOC,  or  ramps  down  as  the  system  is  retired.  Since  the  expectation  is  to  provide  full 
capability  it  is  appropriate  to  neglect  any  partial  capability  for  this  trade. 

b.  Model  Output 

The  output  of  the  mathematical  model  is  the  “Cost  per  Year  per  OMOE” 
for  each  communications  satellite  system  architecture  evaluated.  The  lower  the  cost  per 
year  per  OMOE  per  year,  the  more  preferred  the  system  architecture  is.  Final  selection  of 
the  appropriate  system  architecture  by  decision  makers  is  expected  to  be  based  on  more 
than  just  the  results  of  this  mathematical  model.  Other  factors  outside  of  the  model  will 
need  to  be  considered  in  the  final  satellite  system  architecture  selection  process.  Such 
factors  may  include  budget  constraints,  rapidly  evolving  threats  and  needs,  and  the  level 
of  program  risk.  However,  the  model  output  will  provide  objective  data  regarding 
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requirements  satisfaction,  which  will  help  decision  makers  make  an  infonned  selection 
regarding  which  system  architecture  to  develop,  produce,  or  maintain. 

Additional  post  processing  of  the  cost  per  year  per  OMOE  result  can  be 
perfonned  for  the  display  of  the  results  or  sensitivity  analysis.  The  data  post  processing 
efforts  are  defined  by  the  desires  and  expectations  of  the  decision  makers  utilizing  the 
data.  The  development  of  standard  post  processing  efforts  is  beyond  the  scope  of  this 
thesis. 


c.  Model  Calculations 

The  following  is  a  description  of  how  the  inputs  are  transformed  into  an 
output  in  the  mathematical  model.  The  approach  described  can  easily  be  implemented  in 
Microsoft  Excel,  or  any  other  calculation  software.  The  number  of  key  attributes,  n,  is  a 
positive  integer.  For  making  a  comparison,  the  number  of  system  architectures  is 
necessarily  an  integer  greater  than  one.  The  steps  demonstrated  are  for  a  given 
architecture  and  must  be  repeated  for  every  system  architecture  under  consideration. 

The  first  step  is  to  calculate  a  Raw  Score  that  reflects  the  level  of  achieved 
perfonnance  for  each  key  attribute.  This  is  done  by  computing  the  ratio  of  achieved 
perfonnance  to  the  range  as  shown  in  Equation  1 . 


„  „  Achieved  Performance  -  Threshold  Requirement  , , . 

Raw  Score  = - - -  (1) 

Objective  Requirement  -  Threshold  Requirement 


The  achieved  perfonnance  of  all  key  attributes  must  be  equal  to  or  greater 
than  the  threshold  performance  level.  As  mentioned  earlier,  a  system  with  an  achieved 
perfonnance  below  the  threshold  will  only  be  considered  if  the  decision  maker  sets  a  new 
threshold  level  for  all  architectures  and  the  model  is  updated  with  the  new  threshold 
level.  Otherwise,  the  architecture  will  be  omitted  from  consideration. 

Next,  the  Raw  Score  must  be  conected  so  that  the  maximum  value  of  the 
attribute  Score  is  1.0.  This  discourages  obtaining  greater  than  objective  level  perfonnance 
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at  the  expense  of  other  key  attributes.  Any  Raw  Score  values  that  Equation  1  calculates 
higher  than  1.0  are  automatically  assigned  a  value  of  1.0  and  represents  the  Corrected 
Score  that  is  used  for  future  calculations. 

Once  all  Raw  Score  values  are  calculated  and  comply  with  the  two  rules, 
the  Weighted  Score  for  each  attribute  and  system  architecture  is  computed.  This  is  done 
by  calculating  the  product  of  the  Corrected  Performance  Score  and  the  input  attribute 
weighting  as  shown  in  Equation  2. 


Weighted  Score  =  Corrected  Performance  Score  x  Attribute  Weight  (2) 

With  the  Weighted  Scores  of  all  key  attributes  calculated,  the  OMOE 
score  for  each  system  architecture  can  be  computed.  For  the  number  of  key  attributes,  n, 
the  OMOE  score  is  the  sum  of  all  Weighted  Scores  of  the  system  architecture  as  shown  in 
Equation  3.  There  is  an  OMOE  score  for  each  system  architecture. 


Number  Key  Attributes 

OMOE  Score  =  ^  Weighted  Scoren  (3) 

n=l 


Finally,  the  cost  per  year  per  OMOE  is  calculated  for  each 
communications  satellite  system  architecture  by  dividing  the  total  system  architecture 
cost  by  number  of  years  the  system  is  planned  to  be  FOC  and  the  computed  total  OMOE 
score.  This  is  shown  in  Equation  4.  The  total  system  cost  includes  all  development, 
acquisition,  fielding,  operations,  maintenance  and  upgrades,  and  system  disposal  costs 
paid  by  the  Government  or  customer. 


Cost/year/OMOE  = 


Cost 

FOC  Years  x  OMOE  Score 


(4) 
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D.  CHAPTER  SUMMARY 

This  chapter  has  presented  a  survey  of  candidate  system  parameters  upon  which 
to  base  a  mathematical  model  that  can  be  used  to  conduct  trade  studies  on  candidate 
communications  satellite  architectures.  The  five  primary  key  attributes  are: 
communication  capacity,  access,  interoperability,  commandability,  and  infonnation 
assurance  and  protection.  Only  the  capacity  attribute  should  be  considered  necessary  to 
all  system  architecture  trade  analyses,  due  to  the  commonality  with  current  trade  analyses 
and  commercial  leasing  metrics.  Including  key  attributes  not  identified  in  the  primary  list 
are  encouraged  but  must  be  based  on  the  specific  requirements  defined  for  the  system  and 
may  include  KPPs,  TPMs,  and  overarching  requirements. 

Three  mathematical  model  approaches  were  discussed.  The  selected  approach  is  a 
combination  of  two  of  them  and  enables  a  comparison  of  the  cost  for  achieved 
performance  level.  This  approach  leveraged  three  mathematical  model  benefits:  1)  the 
ability  for  stakeholders  to  weight  the  relative  importance  of  all  key  attributes  utilized  in 
the  model;  2)  a  calculation  of  raw  scores  to  measure  the  performance  of  each  key 
attribute  across  system  architectures;  and  3)  consideration  of  the  total  system  cost  for 
each  system  architecture. 

The  inputs,  output,  and  calculation  methodology  of  the  selected  mathematical 
model  were  presented.  The  output  of  the  mathematical  model  is  a  cost  per  year  per 
OMOE  score.  The  OMOE  score  multiplied  by  the  number  of  years  at  Full  Operational 
Capability  can  be  considered  a  unit  of  effectiveness.  The  higher  the  OMOE,  the  more 
effective  the  system  architecture  is  at  meeting  requirement  objectives.  The  longer  a 
system  can  be  at  FOC,  the  more  effective  the  system  architecture  is  at  providing 
communications  capability.  Thus,  dividing  the  real  cost  of  a  system  by  a  unit  of 
effectiveness  generates  an  “effective  cost.”  The  result  is  an  intuitive  metric  for  decision 
makers  to  utilize  in  their  system  architecture  selection.  Namely,  the  most  preferred 
system  architecture  will  have  the  lowest  cost  per  unit  of  effectiveness.  We  cannot  claim 
“best,”  as  the  decision  makers  will  still  have  to  consider  these  results  in  concert  with 
other  factors  that  are  not  quantitatively  included  in  the  model,  such  as  how  soon  a  system 
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can  be  fielded,  budget  constraints,  and  overall  level  of  program  risk  before  making  a  final 
communication  satellite  system  architecture  selection. 
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III.  USE  OF  THE  MATHEMATICAL  MODEL 


A  mathematical  model  for  conducting  trade  studies  on  candidate  communications 
satellite  architectures  has  been  presented.  In  this  chapter  the  mathematical  model  will  be 
exercised  to  demonstrate  the  functionality.  The  satellite  systems,  requirements,  and 
performance  inputs  used  in  this  chapter  are  for  illustrative  purposes  only,  and  do  not 
represent  real  or  planned  satellite  systems. 

The  system  architecture  envisioned  in  this  example  is  intended  to  provide 
communications  capability  world-wide  between  65°  north  and  south  latitude.  The 
planned  usage  of  the  communications  capability  is  described  in  a  DoD  generated  scenario 
that  includes  details  about  tenninal  types,  locations,  and  data  needs  in  both  nominal  and 
stressed  contingencies.  This  use  of  a  scenario  to  quantify  the  capacity  provided  by  the 
satellite  communications  architecture  is  the  same  process  utilized  traditionally  for 
assessing  systems.  To  provide  worldwide  coverage,  the  space  system  envisioned  will 
utilize  a  minimum  of  four  satellites  in  geosynchronous  orbit.  There  is  no  restriction  on 
the  use  of  more  satellites  to  meet  system  requirements. 

A.  DEFINITION  OF  THE  SET  OF  KEY  ATTRIBUTES  FOR  THE 

SATELLITE  COMMUNICATIONS  SYSTEM  ARCHITECTURE 

The  first  steps  in  creating  the  mathematical  model  are  to  select  and  weight  the  key 
attributes  of  the  communications  satellite  system  architecture.  There  is  no  upper  limit  to 
the  number  of  attributes  that  the  model  can  support;  however,  the  list  must  be 
manageable  and  understandable  to  the  stakeholders.  For  this  exercise,  all  five  of  the 
primary  attributes  identified  previously  will  be  utilized  in  conjunction  with  six  additional 
requirements.  These  attributes  along  with  their  threshold  and  objective  values  will  be 
discussed  below.  All  attributes  are  weighted  equally,  except  for  Communications 
Capacity  and  the  Full  Operational  Capability  date,  which  have  double  the  weight  of  the 
other  attributes.  Assigning  a  different  weight  to  these  two  attributes  demonstrates  the 
flexibility  the  user  has  in  assigning  priorities  to  the  attributes  under  consideration. 
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1.  Communications  Capacity 

Communications  Capacity  is  the  total  throughput  supported  by  the  system 
architecture  in  a  specified  scenario  that  includes  the  global  types,  locations,  and  data 
needs.  The  units  for  this  attribute  are  Gigabits  per  second  (Gbps).  The  threshold  value  is 
15  Gbps  and  the  objective  value  is  50  Gbps. 

2.  Nominal  Case  Access 

The  Nominal  Case  Access  attribute  represents  the  percentage  of  tenninals  located 
globally  that  can  be  supported  in  the  specified  scenario  that  represents  expected  nominal 
usage.  This  would  include  all  requirements  for  both  operations  and  training  exercises. 
This  attribute  is  included  in  addition  to  the  system  capacity  to  limit  biasing  the 
performance  score  by  choosing  to  support  only  a  few  high  capacity  terminals  and 
abandoning  the  support  of  numerous  smaller  tenninals.  The  units  for  the  attribute  are 
percent.  The  threshold  value  is  20%  and  the  objective  value  is  100%. 

3.  Stressed  Case  Access 

The  Stressed  Case  Access  attribute  represents  the  percentage  of  tenninals  located 
globally  that  can  be  supported  in  a  specified  scenario  that  represents  expected  stressed 
usage.  In  a  stressed  situation  the  layout  of  terminals  is  expected  to  be  different,  including 
higher  concentrations  of  tenninals  than  the  Nominal  Case  Access  model  in  some 
geographically  distinct  areas.  This  attribute  is  included  in  addition  to  the  system  capacity 
is  to  limit  biasing  the  performance  scores  by  choosing  to  support  only  a  few  high  capacity 
terminals,  and  abandoning  the  support  of  numerous  smaller  terminals.  The  units  for  the 
attribute  are  percent.  The  threshold  value  is  30%,  and  the  objective  value  is  100%. 

4.  Interoperability 

Interoperability  quantifies  the  extent  to  which  different  user  terminal  pairs  can 
operate  together.  System  requirements  support  communication  in  four  distinct  frequency 
bands,  C,  X,  Ku,  and  Ka,  for  example.  Communication  between  terminals  of  the  same 
frequency  band  is  automatically  accomplished;  however,  it  is  highly  desirable  to  have  the 
ability  to  have  unmatched  frequency  band  tenninals  interface  seamlessly.  This  means  that 
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the  satellite  converts  between  bands,  for  example  receiving  a  C  band  input  from  one 
tenninal  and  linking  that  data  stream  to  a  receiving  tenninal  in  the  Ku  band.  It  is  possible 
to  support  some  of  this  capability  on  the  ground  with  a  multi  band  tenninal,  but  if  the 
functionality  can  be  accommodated  on  the  satellite  at  a  reasonable  price  this  would 
eliminate  operations  and  maintenance  constraints  and  costs  to  the  tenninals.  This  attribute 
is  the  count  of  the  number  of  bands  that  can  be  converted  from  one  to  another.  The 
threshold  value  is  0  and  the  objective  value  is  12.  This  input  is  an  integer  value. 

5.  Commandability 

The  Commandability  attribute  focuses  on  the  autonomous  response  of  each 
satellite  in  the  constellation  by  quantifying  how  long  the  system  will  continue  to  provide 
communication  service  in  the  last  commanded  configuration.  The  configuration  includes 
providing  satellite  station  keeping,  beam  pointing,  bandwidth  allocations,  and  the  system 
event  logs  that  can  be  downloaded  when  ground  control  is  restored.  The  threshold  value 
is  5  days  and  the  objective  value  is  20  days. 

6.  Information  Protection 

The  Information  Protection  attribute  will  measure  compliance  with  current 
information  assurance  and  infonnation  protection  policies.  Waivers  from  the  list  of 
existing  policies  that  must  be  complied  with  are  possible  but  must  be  minimized.  This 
attribute  will  count  the  number  of  waivers  that  must  be  obtained.  The  threshold  value  is  5 
waivers  and  the  objective  is  0  waivers.  This  input  is  an  integer  value. 

7.  Initial  Operational  Capability  Date 

This  attribute  tracks  when  Initial  Operational  Capability  (IOC)  is  expected  to  be 
achieved.  The  threshold  value  is  2025  and  the  objective  is  2018.  This  input  is  an  integer 
value. 
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8. 


Full  Operational  Capability  Date 


This  attribute  tracks  when  Full  Operational  Capability  (FOC)  is  expected  to  be 
achieved.  The  threshold  value  is  2030  and  the  objective  is  2024.  This  input  is  an  integer 
value. 


9.  Constellation  Restoration  Time 

The  Constellation  Restoration  Time  attribute  quantifies  how  long  it  takes  to 
restore  FOC  when  a  satellite  in  the  constellation  suffers  a  catastrophic  loss.  This  allows  a 
trade  of  constellation  capability  based  on  whether  a  spare  is  a)  only  manufactured  and 
launched  after  a  failure  occurs;  b)  manufactured  ahead  of  time  and  launched  after  the 
failure  occurs;  or  c)  maintained  on-orbit  and  available  to  be  repositioned  after  the  failure 
occurs.  The  threshold  value  is  5  years  and  the  objective  value  is  0.25  years. 

10.  Launch  Vehicle  Compatibility 

This  attribute  addresses  the  flexibility  in  launch  vehicles  available  for  use. 
Flexibility  in  the  launch  vehicle  used  can  enable  a  more  robust  fielding  the  system  in  the 
event  of  any  production  issues  of  the  launch  vehicle.  The  threshold  value  is  1  launch 
vehicle  and  the  objective  is  3  different  launch  vehicles.  This  value  is  an  integer. 

11.  Anti-Jam  Capability  Level 

The  Anti- Jam  Capability  Level  attribute  indicates  the  combination  of  Anti- Jam 
capabilities  provided  by  the  communications  satellite  system  architecture.  The  levels  are 
integer  values  that  indicate  compliance  with  a  variety  of  requirement  levels  including 
interference  signals;  interference  nulling;  geolocation  identification  and  reporting;  and 
the  amount  of  power  margin  against  an  interference  beam.  Level  1  represents  the  most 
robust  Anti-Jam  capability  required  based  on  the  combined  performance  of  the 
constituent  requirements.  For  example  a  Level  1  score  would  incorporate  a  high  level  of 
adaptive  processing  to  isolate  interference  signals,  nulling  capability  for  the  quantity  and 
locations  of  interference  sources,  the  ability  to  identify  and  report  the  location  of 
interference  sources  to  the  required  accuracy,  and  the  ability  to  successfully  transmit  in 
the  presence  of  interference.  Level  2  would  represent  the  next  best  Anti-Jam  capability 
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achievement,  and  so  forth  to  level  5,  which  would  represent  the  minimum  acceptable 
level  of  Anti-Jam  capability.  The  threshold  value  is  5  and  the  objective  value  is  1.  This 
value  is  an  integer. 

B.  DEFINITION  OF  A  SET  OF  ARCHITECTURES 

For  this  example  a  set  of  nine  fictitious  communications  satellite  system 
architectures  were  analyzed.  Each  architecture  represents  a  concept  that  meets  the 
requirements  through  a  different  approach  such  as  would  be  generated  by  multiple 
contractors.  The  range  of  system  performance  levels  may  result  from  varying: 
complexity  and  capability  of  the  individual  satellites;  technology  and  production 
development  times;  satellite  production  rates;  and  levels  of  control  and  autonomy  of  the 
fielded  system. 

As  with  the  key  attributes,  there  is  no  limit  to  the  number  of  system  architectures 
that  can  be  analyzed  by  the  mathematical  model.  A  set  of  nine  architectures  was  selected 
to  exercise  the  model  in  this  example.  The  model  and  data  have  been  input  in  a  Microsoft 
Excel  2007  Workbook.  This  exercise  is  not  based  on  a  real  system;  therefore,  values 
assigned  for  each  key  attribute  were  assigned  by  a  random  number  generator.  None  of  the 
performance  inputs  were  allowed  to  be  below  the  threshold  value.  Perfonnance  levels 
above  the  objective  value  were  allowed  when  appropriate  to  verify  that  the  Excel  model 
properly  corrects  the  attribute  scores.  Perfonnance  above  the  objective  level  is  not 
realistic  for  the  number  of  Infonnation  Protection  waivers,  Anti-Jam  Capability  Level, 
and  the  percentage  of  terminals  supported  attributes. 

C.  EVALUATION  OF  A  SET  OF  ARCHITECTURES  USING  THE 

MATHEMATICAL  MODEL 

The  mathematical  model  was  implemented  using  an  Excel  spreadsheet  with  the 
defined  inputs  utilized  in  the  model. 

The  weighting  for  the  key  attributes  is  shown  in  Table  l.A  set  of  conditional 

fonnatting  rules  is  applied  to  the  Weighting  Check  value  such  that  if  the  cell  has  a  value 

of  one  the  cell  fill  will  be  green.  If  the  cell  value  is  greater  or  less  than  1  the  Weighting 

Check  box  will  turn  red  to  indicate  an  erroneous  input.  This  check  is  illustrated  when  the 
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Constellation  Restoration  Time  weight  is  set  to  zero  as  shown  in  Table  2.  The  key 
attributes  for  Communications  Capacity  and  FOC  Date  were  selected  to  be  more 
important  than  all  other  attributes,  reflected  by  a  weight  that  is  twice  the  value  of  the 
other  weights.  Distributing  the  weights  to  the  key  attributes  per  this  relative  importance 
results  in  the  distribution  shown  in  Table  1. 


Table  1.  Mathematical  Model  Weighting  Inputs. 


Key  Attributes 

Weight 

1.  Capacity  (Gbps) 

0.15 

2.  Nominal  Case  Access  (%  terminals  supported) 

0.08 

3.  Stressed  Case  Access  (%  terminals  supported) 

0.08 

4.  Interoperability  (#  of  cross  band  conversions) 

0.08 

5.  Commandability  (days  of  autonomous  operations) 

0.08 

6.  Information  Protection  (#  of  policy  waivers) 

0.08 

7.  IOC  Date  (year) 

0.08 

8.  FOC  date  (year) 

0.15 

9.  Constellation  Restoration  Time  (years) 

0.08 

10.  Launch  Vehicle  Flexibility 

0.08 

11.  Anti  Jam  Capability  Level 

0.08 

Weighting  check 

1.00 

Table  2.  Mathematical  Model  Weighting  Check  Error. 


Key  Attributes 

Weight 

1.  Capacity  (Gbps) 

0.15 

2.  Nominal  Case  Access  (%  terminals  supported) 

0.08 

3.  Stressed  Case  Access  (%  terminals  supported) 

0.08 

4.  Interoperability  (#  of  cross  band  conversions) 

0.08 

5.  Commandability  (days  of  autonomous  operations) 

0.08 

6.  Information  Protection  (#  of  policy  waivers) 

0.08 

7.  IOC  Date  (year) 

0.08 

8.  FOC  date  (year) 

0.15 

9.  Constellation  Restoration  Time  (years) 

0.00 

10.  Launch  Vehicle  Flexibility 

0.08 

11.  Anti  Jam  Capability  Level 

0.08 

Weighting  check 
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The  threshold  and  objective  values  for  each  key  attribute  are  shown  in  Table  3. 
Additionally,  the  table  shows  whether  an  attribute  is  to  be  minimized  or  maximized.  If 
the  objective  value  is  larger  than  the  threshold  value  the  attribute  is  to  be  maximized; 
otherwise,  the  attribute  is  to  be  minimized.  The  Criteria  Type  result  is  used  to  error  check 
Table  4. 


Table  3.  Mathematical  Model  Threshold  and  Performance  Inputs. 


1.  Capacity 
(Gbps) 

2.  Nominal 

Case 

Access  (% 
terminals 
supported) 

3.  Stressed 

Case 

Access  (% 
terminals 
supported) 

4.  Interoper¬ 
ability  (#of 
cross  band 
conversions) 

5.  Command- 
ability  (days 
of 

autonomous 

operations) 

6.  Information 
Protection  (# 
of  policy 
waivers) 

7.  IOC  Date 
(year) 

8.  FOC  date 
(year) 

9.  Constellation 

Restoration 
Time  (years) 

10.  Launch 

Vehicle 

Flexibility 

11.  Anti-Jam 
Capability 
Level 

Threshold 

15 

0.2 

0.3 

0 

5 

5 

2025 

2030 

5 

1 

5 

Objective 

50 

1 

1 

12 

20 

0 

2018 

2024 

0.25 

3 

1 

Criteria  Type 

maximize 

maximize 

maximize 

maximize 

maximize 

minimize 

minimize 

minimize 

minimize 

maximize 

minimize 

The  perfonnance  inputs  generated  by  the  random  number  generator  for  each 
attribute  are  shown  in  Table  4.  The  green  highlight  in  the  cell  for  each  perfonnance  input 
indicates  the  requirement  for  the  value  to  meet  or  exceed  the  threshold  requirement  has 
been  met.  This  is  accomplished  by  first  comparing  the  performance  input  value  to  the 
threshold  requirement.  If  the  key  attribute  value  is  to  be  maximized  to  meet  objective 
performance,  the  input  must  be  equal  to  or  greater  than  the  threshold  value.  If  the  key 
attribute  value  is  to  be  minimized  to  meet  objective  performance,  the  input  must  be  equal 
to  or  less  than  the  threshold  value.  This  check  is  accomplished  in  Excel  by  using  the 
Excel  IF  function.  The  results  of  the  performance  check  are  indicated  in  a  calculation 
section  of  the  spreadsheet  with  a  “Pass”  or  “Threshold  Error.”  Table  5  shows  the  results 
of  this  intermediate  calculation  for  the  model  inputs.  The  highlight  in  the  performance 
input  cell  is  accomplished  by  utilizing  two  Excel  conditional  formatting  rules,  one  that 
colors  the  cell  green  if  the  calculation  sheet  has  a  “Pass”  in  the  corresponding  system 
attribute  cell,  and  the  second  to  color  the  cell  red  if  the  calculation  sheet  has  a  “Threshold 
Error”  in  the  corresponding  system  attribute  cell.  Calculations  cannot  continue  until  the 
threshold  error  is  resolved  by  eliminating  the  system  from  consideration,  adjusting  the 
threshold  value  requirement  so  that  all  systems  pass,  or  adjusting  the  system  design  so 
that  the  threshold  requirement  can  be  met. 
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Functionality  of  each  cell’s  threshold  check  was  verified.  An  example  of  this 
functionality  check  is  illustrated  in  Tables  6  and  7  where  System  Concept  3  Capacity, 
System  Concept  4  Commandability,  and  System  Concept  7  Anti-Jam  Capability  Level 
were  all  set  below  the  threshold  level.  These  values  were  not  used  in  further  model 
calculations. 


Table  4.  Mathematical  Model  Perfonnance  Inputs. 


1.  Capacity 
(Gbps) 

2.  Nominal 

Case 

Access  (% 
terminals 
supported) 

3.  Stressed 

Case 

Access  (% 
terminals 
supported) 

4.  Interoper¬ 
ability  (#of 
cross  band 
conversions) 

5.  Command- 
ability  (days 
of 

autonomous 

operations) 

6.  Information 
Protection  (# 
of  policy 
waivers) 

7.  IOC  Date 
(year) 

8.  FOC  date 
(year) 

9.  Constellation 

Restoration 
Time  (years) 

10.  Launch 

Vehicle 

Flexibility 

11.  Anti-Jam 
Capability 
Level 

System  Concept  1 

41.95 

0.31 

0.91 

3 

13 

5 

2021 

2024 

5 

4 

2 

System  Concept  2 

20.35 

0.95 

0.77 

14 

20 

5 

2018 

2027 

0.75 

2 

3 

System  Concept  3 

16.89 

0.6 

0.75 

4 

11 

4 

2019 

2030 

4 

3 

4 

System  Concept  4 

17.16 

0.72 

0.89 

8 

22 

2 

2018 

2024 

1 

2 

1 

System  Concept  5 

38.05 

0.4 

0.42 

5 

20 

4 

2018 

2028 

3.25 

3 

1 

System  Concept  6 

24.24 

0.33 

0.37 

3 

10 

1 

2024 

2029 

1.25 

3 

1 

System  Concept  7 

38.85 

0.62 

0.52 

0 

6 

3 

2023 

2029 

2.75 

4 

1 

System  Concept  8 

28.34 

0.96 

0.63 

4 

18 

5 

2022 

2029 

1.25 

4 

4 

System  Concept  9 

15.84 

0.9 

0.63 

13 

20 

4 

2018 

2027 

1.5 

4 

2 

Table  5.  Mathematical  Model  Threshold  Error  Check  Intermediate  Calculation. 


1.  Capacity 
(Gbps) 

2.  Nominal 

Case 

Access  (% 
terminals 
supported) 

3.  Stressed 

Case 

Access  (% 
terminals 
supported) 

4.  Interoper¬ 
ability  (#  of 
cross  band 
conversions) 

5.  Command- 
ability  (days 
of 

autonomous 

operations) 

6.  Information 
Protection  (# 
of  policy 
waivers) 

7.  IOC  Date 
(year) 

8.  FOC  date 
(year) 

9.  Constellation 

Restoration 
Time  (years) 

10.  Launch 

Vehicle 

Flexibility 

11.  Anti-Jam 
Capability 
Level 

System  Concept  1 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  2 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  3 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  4 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  5 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  6 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  7 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  8 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  9 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Table  6.  Mathematical  Model  Input  Threshold  Check  Validation  Example. 


1.  Capacity 
(Gbps) 

2.  Nominal 

Case 

Access  (% 
terminals 
supported) 

3.  Stressed 

Case 

Access  (% 
terminals 
supported) 

4.  Interoper¬ 
ability  (#  of 
cross  band 
conversions) 

5.  Command- 
ability  (days 
of 

autonomous 

operations) 

6.  Information 
Protection  (ft 
of  policy 
waivers) 

7.  IOC  Date 

(year) 

8.  FOC  date 
(year) 

9.  Constellation 

Restoration 

Time  (years) 

10.  Launch 

Vehicle 

Flexibility 

11.  Anti-Jam 
Capability 
Level 

System  Concept  1 

41.95 

0.31 

0.91 

3 

13 

5 

2021 

2024 

5 

4 

2 

System  Concept  2 

20.35 

0.95 

0.77 

14 

20 

5 

2018 

2027 

0.75 

2 

3 

System  Concept  3 

0.6 

0.75 

4 

11 

4 

2019 

2030 

4 

3 

4 

System  Concept  4 

17.16 

0.72 

0.89 

8 

2 

2018 

2024 

1 

2 

1 

System  Concept  5 

38.05 

0.4 

0.42 

5 

20 

4 

2018 

2028 

3.25 

3 

1 

System  Concept  6 

24.24 

0.33 

0.37 

3 

10 

1 

2024 

2029 

1.25 

3 

1 

System  Concept  7 

38.85 

0.62 

0.52 

0 

6 

3 

2023 

2029 

2.75 

4 

System  Concept  8 

28.34 

0.96 

0.63 

4 

18 

5 

2022 

2029 

1.25 

4 

4 

System  Concept  9 

15.84 

0.9 

0.63 

13 

20 

4 

2018 

2027 

1.5 

4 

2 
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Table  7.  Mathematical  Model  Threshold  Error  Check  Intermediate  Calculation  Validation 

Example. 


1.  Capacity 
(Gbps) 

2.  Nominal 

Case 

Access  (% 
terminals 
supported) 

3.  Stressed 

Case 

Access  (% 
terminals 
supported) 

4.  Interoper¬ 
ability  (#of 
cross  band 
conversions) 

5.  Command- 
ability  (days 
of 

autonomous 

operations) 

6.  Information 
Protection  (# 
of  policy 
waivers) 

7.  IOC  Date 
(year) 

8.  FOC  date 
(year) 

9.  Constellation 

Restoration 
Time  (years) 

10.  Launch 

Vehicle 

Flexibility 

11.  Anti-Jam 
Capability 
Level 

System  Concept  1 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  2 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  3 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  4 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  5 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  6 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  7 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  8 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

System  Concept  9 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

Pass 

The  remaining  inputs  are  the  system  cost  and  number  of  years  at  FOC.  For  the 
example  all  systems  will  be  at  FOC  for  10  years.  This  fixed  FOC  period  could  represent  a 
situation  where  the  next  generation  system  is  required  to  be  fielded.  The  total  cost  for 
each  satellite  communication  system  architecture  was  generated  as  a  random  number 
between  $3B  and  $6B.  The  inputs  for  these  parameters  are  shown  in  Table  8. 


Table  8.  Mathematical  Model  Cost  and  FOC  Duration  Inputs. 


System  Architecture 

Cost  ($B) 

Years  at  FOC 

System  Concept  1 

3.6 

10 

System  Concept  2 

3.3 

10 

System  Concept  3 

4.5 

10 

System  Concept  4 

6.0 

10 

System  Concept  5 

5.7 

10 

System  Concept  6 

3.8 

10 

System  Concept  7 

5.1 

10 

System  Concept  8 

4.7 

10 

System  Concept  9 

4.7 

10 

The  calculation  of  the  OMOE  score  occurs  in  three  steps  in  the  spreadsheet.  First, 
an  uncorrected  raw  perfonnance  score  is  computed  for  each  attribute  using  Equation  1. 
The  results  of  this  computation  are  shown  in  Table  9.  This  example  problem  included 
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some  performance  inputs  that  exceeded  the  objective  level,  resulting  in  a  raw 
perfonnance  score  that  is  greater  than  1.0,  such  as  the  Interoperability  for  System  2  and 
the  Commandability  for  System  4. 


Table  9.  Raw  Performance  Calculation  Results. 


1.  Capacity 
(Gbps) 

2.  Nominal 

Case 

Access  (% 
terminals 
supported) 

3.  Stressed 

Case 

Access  (% 
terminals 
supported) 

4.  Interoper¬ 
ability  (#of 
cross  band 
conversions) 

5.  Command- 
ability  (days  of 

autonomous 

operations) 

6.  Information 
Protection  (# 
of  policy 
waivers) 

7.  IOC  Date 
(year) 

8.  FOC  date 
(year) 

9.  Constellation 

Restoration 
Time  (years) 

10.  Launch 

Vehicle 

Flexibility 

11.  Anti-Jam 
Capability 
Level 

System  Concept  1 

0.77 

0.14 

0.87 

0.25 

0.53 

0.00 

0.57 

1.00 

0.00 

1.50 

0.75 

System  Concept  2 

0.15 

0.94 

0.67 

1.17 

1.00 

0.00 

1.00 

0.50 

0.89 

0.50 

0.50 

System  Concept  3 

0.05 

0.50 

0.64 

0.33 

0.40 

0.20 

0.86 

0.00 

0.21 

1.00 

0.25 

System  Concept  4 

0.06 

0.65 

0.84 

0.67 

1.13 

0.60 

1.00 

1.00 

0.84 

0.50 

1.00 

System  Concept  5 

0.66 

0.25 

0.17 

0.42 

1.00 

0.20 

1.00 

0.33 

0.37 

1.00 

1.00 

System  Concept  6 

0.26 

0.16 

0.10 

0.25 

0.33 

0.80 

0.14 

0.17 

0.79 

1.00 

1.00 

System  Concept  7 

0.68 

0.53 

0.31 

0.00 

0.07 

0.40 

0.29 

0.17 

0.47 

1.50 

1.00 

System  Concept  8 

0.38 

0.95 

0.47 

0.33 

0.87 

0.00 

0.43 

0.17 

0.79 

1.50 

0.25 

System  Concept  9 

0.02 

0.88 

0.47 

1.08 

1.00 

0.20 

1.00 

0.50 

0.74 

1.50 

0.75 

The  next  calculation  step  is  the  correction  of  the  raw  performance  scores  to  limit 
the  score  to  a  maximum  of  one.  If  the  raw  perfonnance  score  is  greater  than  1,  it  is 
automatically  reset  to  1  to  prohibit  systems  from  benefiting  from  exceeding  objective 
performance.  The  result  of  this  correction  is  shown  in  Table  10. 


Table  10.  Corrected  Performance  Score  Calculation  Results. 


1.  Capacity 
(Gbps) 

2.  Nominal 

Case 

Access  (% 
terminals 
supported) 

3.  Stressed 

Case 

Access  (% 
terminals 
supported) 

4. 

Interoperabilit 
y  (#  of  cross 
band 

conversions) 

5. 

Commandabili 
ty  (days  of 

autonomous 

operations) 

6.  Information 
Protection  (# 
of  policy 
waivers) 

7.  IOC  Date 
(year) 

8.  FOC  date 
(year) 

9.  Constellation 

Restoration 
Time  (years) 

10.  Launch 

Vehicle 

Flexibility 

11.  Anti-Jam 
Capability 
Level 

System  Concept  1 

0.77 

0.14 

0.87 

0.25 

0.53 

0.00 

0.57 

1.00 

0.00 

1.00 

0.75 

System  Concept  2 

0.15 

0.94 

0.67 

1.00 

1.00 

0.00 

1.00 

0.50 

0.89 

0.50 

0.50 

System  Concept  3 

0.05 

0.50 

0.64 

0.33 

0.40 

0.20 

0.86 

0.00 

0.21 

1.00 

0.25 

System  Concept  4 

0.06 

0.65 

0.84 

0.67 

1.00 

0.60 

1.00 

1.00 

0.84 

0.50 

1.00 

System  Concept  5 

0.66 

0.25 

0.17 

0.42 

1.00 

0.20 

1.00 

0.33 

0.37 

1.00 

1.00 

System  Concept  6 

0.26 

0.16 

0.10 

0.25 

0.33 

0.80 

0.14 

0.17 

0.79 

1.00 

1.00 

System  Concept  7 

0.68 

0.53 

0.31 

0.00 

0.07 

0.40 

0.29 

0.17 

0.47 

1.00 

1.00 

System  Concept  8 

0.38 

0.95 

0.47 

0.33 

0.87 

0.00 

0.43 

0.17 

0.79 

1.00 

0.25 

System  Concept  9 

0.02 

0.88 

0.47 

1.00 

1.00 

0.20 

1.00 

0.50 

0.74 

1.00 

0.75 

The  weighted  score  for  each  performance  input  is  calculated  by  applying 
Equation  2.  The  OMOE  score  is  calculated  by  applying  Equation  3.  The  results  of  both 
these  calculations  are  shown  in  Table  11.  The  OMOE  score  is  highlighted  in  blue. 
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Table  11.  Weighted  Performance  Scores  and  OMOE  Calculation  Results. 


1.  Capacity 
(Gbps) 

2.  Nominal 

Case 

Access  (% 
terminals 
supported) 

3.  Stressed 

Case 

Access  (% 
terminals 
supported) 

4.  Interoper¬ 
ability  (#  of 
cross  band 
conversions) 

5.  Command- 
ability  (days  of 

autonomous 

operations) 

6.  Information 
Protection  (# 
of  policy 
waivers) 

7.  IOC  Date 
(year) 

8.  FOC  date 
(year) 

9.  Constellation 

Restoration 
Time  (years) 

10.  Launch 

Vehicle 

Flexibility 

11.  Anti-Jam 
Capability 
Level 

OMOE  Score 

System  Concept  1 

0.12 

0.01 

0.07 

0.02 

0.04 

0.00 

0.04 

0.15 

0.00 

0.08 

0.06 

0.59 

System  Concept  2 

0.02 

0.07 

0.05 

0.08 

0.08 

0.00 

0.08 

0.08 

0.07 

0.04 

0.04 

0.60 

System  Concept  3 

0.01 

0.04 

0.05 

0.03 

0.03 

0.02 

0.07 

0.00 

0.02 

0.08 

0.02 

0.35 

System  Concept  4 

0.01 

0.05 

0.06 

0.05 

0.08 

0.05 

0.08 

0.15 

0.06 

0.04 

0.08 

0.71 

System  Concept  5 

0.10 

0.02 

0.01 

0.03 

0.08 

0.02 

0.08 

0.05 

0.03 

0.08 

0.08 

0.57 

System  Concept  6 

0.04 

0.01 

0.01 

0.02 

0.03 

0.06 

0.01 

0.03 

0.06 

0.08 

0.08 

0.42 

System  Concept  7 

0.10 

0.04 

0.02 

0.00 

0.01 

0.03 

0.02 

0.03 

0.04 

0.08 

0.08 

0.44 

System  Concept  8 

0.06 

0.07 

0.04 

0.03 

0.07 

0.00 

0.03 

0.03 

0.06 

0.08 

0.02 

0.48 

System  Concept  9 

0.00 

0.07 

0.04 

0.08 

0.08 

0.02 

0.08 

0.08 

0.06 

0.08 

0.06 

0.62 

Applying  Equation  4,  the  Cost  per  OMOE  per  year  at  FOC  is  calculated  using  the 
OMOE  score  and  the  values  in  Table  8. The  Excel  “Rank”  function  is  applied  to  this 
result  to  graphically  show  the  results,  with  the  highest  rank  associated  with  the  lowest 
Cost  per  year  at  FOC  per  OMOE  and  indicated  in  the  Excel  generated  graded  color  scale 
where  the  most  preferred  system  is  highlighted  in  green  and  least  preferred  is  highlighted 
in  red.  The  results  of  these  calculations  are  shown  in  Table  12. 


Table  12.  Cost  per  OMOE  per  year  at  FOC  Calculation  and  Ranking  Results. 


System  Architecture 

Cost  ($B) 

Years  at  FOC 

OMOE  Score 

Cost/OMOE/FOCyear 

Rank 

System  Concept  1 

3.6 

10 

0.59 

0.61 

2 

System  Concept  2 

3.3 

10 

0.60 

0.55 

1 

System  Concept  3 

4.5 

10 

0.35 

1.30 

9 

System  Concept  4 

6.0 

10 

0.71 

0.85 

4 

System  Concept  5 

5.7 

10 

0.57 

1.00 

7 

System  Concept  6 

3.8 

10 

0.42 

0.91 

5 

System  Concept  7 

5.1 

10 

0.44 

1.15 

8 

System  Concept  8 

4.7 

10 

0.48 

0.99 

6 

System  Concept  9 

4.7 

10 

0.62 

0.76 

3 
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Example  Demonstrating  Threshold-Only  Performance  Handling 

As  mentioned  previously,  there  is  one  special  case  that  may  arise  from  a  given  set 
of  inputs.  In  the  unlikely  event  a  system  achieves  only  threshold  levels  for  all  key 
attributes,  an  OMOE  of  zero  would  result  and  cause  a  divide  by  zero  error.  To  avoid  this, 
the  model  replaces  the  zero  value  with  an  OMOE  of  one  half  the  lowest  OMOE  of  all 
other  system  architectures  being  considered. 

To  demonstrate  how  the  model  handles  this  special  case,  an  additional  system  is 
added  to  the  previous  example.  The  cost  of  the  Threshold  Only  system  architecture  was 
generated  in  the  same  manner  as  for  the  other  system  architectures,  as  a  random  number 
between  $3. OB  and  $6. OB.  In  this  example,  the  cost  of  the  Threshold  Only  system 
architecture  is  $3.9B.Years  at  FOC  matches  the  others  at  10  years.  Table  13  shows  the 
revised  rankings.  In  this  case  the  Threshold  Only  option  is  least  preferred. 


Table  13.  Cost  per  OMOE  per  year  at  FOC  Calculation  and  Updated  Ranking  Results. 


System  Architecture 

Cost  ($B) 

Years  at  FOC 

OMOE  Score 

Cost/OMOE/FOCyear 

Rank 

System  Concept  1 

3.6 

10 

0.59 

0.61 

2 

System  Concept  2 

3.3 

10 

0.60 

0.55 

1 

System  Concept  3 

4.5 

10 

0.35 

1.30 

9 

System  Concept  4 

6.0 

10 

0.71 

0.85 

4 

System  Concept  5 

5.7 

10 

0.57 

1.00 

7 

System  Concept  6 

3.8 

10 

0.42 

0.91 

5 

System  Concept  7 

5.1 

10 

0.44 

1.15 

8 

System  Concept  8 

4.7 

10 

0.48 

0.99 

6 

System  Concept  9 

4.7 

10 

0.62 

0.76 

3 

Threshold  Only 

3.9 

10 

0.17 

2.25 

10  | 

D.  DISCUSSION  OF  RESULTS 

It  is  acknowledged  that  decision  makers  will  still  have  to  consider  the  results  of 
the  model  in  concert  with  other  factors  that  do  not  lend  themselves  to  incorporation  into  a 
mathematical  model,  such  as  budget  constraints,  rapidly  evolving  threats  and  needs,  and 
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the  level  of  program  risk.  An  analysis  of  the  results  will  be  necessary  to  support  the  final 
decision,  with  the  approach  and  focus  to  be  defined  by  the  needs  of  the  decision  makers. 

While  this  analysis  is  based  upon  fictional  inputs,  some  interesting  observations 
can  be  made  about  the  results. 

1.  Comparison  to  Cost  per  Capacity  Approach 

For  comparison  to  the  generally  accepted  current  approach  of  calculating  a  cost 
per  capacity,  the  cost  was  divided  by  the  capacity  defined  in  the  performance  input  for 
each  system.  This  calculation  results  in  a  significantly  different  preference  ranking  of  the 
system  architectures  as  shown  in  Table  14.  The  preference  from  System  Concept  2 
changed  from  1st  to  6th  place.  System  Concept  7  changed  from  8th  to  2nd  place.  The 
Threshold  Only  system  moved  up  from  8th  to  3rd  place.  This  result  indicates  that  when  a 
satellite  communications  system  architecture  has  additional  requirements  beyond  the 
system  capacity,  these  other  key  attributes  must  be  factored  into  the  decision  to  avoid 
eliminating  critical  capabilities  defined  by  the  users. 

This  biasing  can  be  quantifying  by  varying  the  cost  and  capacity  to  identify  when 
the  preference  raking  changes.  System  Concept  1  is  strongly  preferred  in  the  Cost  per 
Capacity  method.  The  capacity  must  drop  from  42  Gbps  to  less  than  28  Gbps,  33%;  or 
the  cost  must  increase  from  $3.6B  to  more  than  $5.5B,  55%,  before  System  Concept  7 
becomes  the  most  preferred  system  architecture.  Decreasing  the  capacity  of  System 
Concept  1  to  less  than  28Gbps  does  not  change  the  2nd  place  preference  in  the  Cost  per 
OMOE  per  year  at  FOC  Calculation.  Increasing  the  cost  of  System  Concept  1  to  $5.6B 
changes  the  preference  from  2nd  to  5th  place.  This  biasing  is  significant,  and  again 
ignores  the  benefits  of  all  other  key  system  requirements,  resulting  in  the  likely  selection 
of  a  system  architecture  solution  that  is  not  optimized  for  either  cost  or  perfonnance. 
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Table  14.  Cost  per  Capacity  Calculation  and  Ranking  Results. 


System  Architecture 

Cost/Capacity 

Cost/Capacity  Rank 

System  Concept  1 

0.08582 

1 

System  Concept  2 

0.16216 

5 

System  Concept  3 

0.26643 

8 

System  Concept  4 

0.34965 

10 

System  Concept  5 

0.14980 

3 

System  Concept  6 

0.15677 

4 

System  Concept  7 

0.13127 

2 

System  Concept  8 

0.16584 

6 

System  Concept  9 

0.29672 

9 

Threshold  Only 

0.26000 

7 

2.  Threshold  Only  Concept  Addition 

The  addition  of  the  Threshold  Only  system  concept  does  not  affect  the  ranking  of 
any  other  system  architecture.  All  other  systems  are  more  preferred.  Adjustment  of  the 
Threshold  Only  system  OMOE  shows  that  the  rankings  do  not  change  until  the  Threshold 
Only  OMOE  is  at  least  87%  of  the  lowest  OMOE  score,  that  of  System  Concept  3.  If  the 
Threshold  Only  system  OMOE  is  made  equal  to  that  of  System  Concept  3  the  ranking  of 
the  Threshold  Only  system  will  increase  to  8th  place  which  remains  well  out  of  range  of 
the  most  preferred  system  concepts.  None  of  the  system  concepts  in  this  example  have 
more  than  one  key  attribute  that  is  at  the  threshold  performance  level,  and  as  such  there  is 
relevant  benefit  to  achievement  of  performance  above  the  threshold  level.  In  light  of  this 
it  would  be  more  appropriate  to  decrease  the  OMOE  score  of  the  threshold  only  system 
relative  to  System  Concept  3.  A  comparison  of  all  Cost  per  OMOE  per  year  at  FOC 
arranged  from  lowest  to  highest  is  shown  in  Figure  1. 
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Figure  1 .  Cost  per  OMOE  per  year  at  FOC  Summary. 


3.  Performance  Score  and  Cost  Comparisons 

The  preference  rankings  clearly  show  that  more  performance  at  any  cost  is  not  the 
best  value  solution.  Cost  is  considered  as  independent  variable  and  accounted  for  in  the 
system  trades.  In  this  example,  we  see  how  system  cost  is  the  dominating  factor  in 
determining  the  preferred  solution.  The  small  cost  or  OMOE  changes  don’t  influence  the 
rankings  significantly. 

The  highest  OMOE  score  belongs  to  System  Concept  4,  the  system  that  is  ranked 
4th  in  preference,  and  is  0.11,  or  18%,  higher  than  the  score  for  System  Concept  2. 
System  Concept  4  is  also  the  most  expensive  system  and  $2.7B,  or  82%,  more  than 
System  Concept  2.  The  82%  cost  difference  dominates  the  18%  performance  benefit. 
Similarly,  for  comparing  System  Concept  9  to  System  Concept  2,  the  2%  OMOE 
perfonnance  benefit  is  dominated  by  the  42%  higher  cost  of  System  Concept  9.  In  both 
instances  cost  will  need  to  be  decreased  for  this  solution  to  become  competitive  with 
System  Concept  2. 

The  OMOE  score  for  System  Concept  2  is  0.02,  or  2%,  higher  than  for  System 
Concept  1,  the  system  ranked  2nd  in  preference.  Additionally,  System  Concept  1,  the 
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system  ranked  2nd  in  preference,  is  $0.3B,  or  9%,  more  than  System  Concept  2.  System 
Concept  2  has  more  capability  at  a  lower  cost,  thus  it  makes  sense  it  is  preferred  over 
system  Concept  1 . 

4.  Performance  Score  and  Cost  Variation 

A  simple  sensitivity  analysis  of  the  cost  input  indicates  that  for  System  Concept  2 
cost  must  increase  by  $0.4B  (12%)  to  $3.7B,  or  the  OMOE  score  must  decrease  by  0.07 
(12%)  to  0.53  to  change  the  preference  rank  from  first  to  second  place.  This  variation  is 
consistent  with  the  differences  described  in  the  previous  section.  Similarly,  a  net  change 
in  cost  and  performance  between  System  Concepts  2  and  9  of  at  least  40%  would  need  to 
be  accomplished  for  System  Concept  9  to  become  the  most  preferred. 

In  an  actual  system  the  cost  and  OMOE  score  are  likely  to  be  related  and 
decreased  cost  will  result  in  decreased  capability.  These  relative  changes,  while  outside 
the  scope  of  this  thesis,  must  be  considered  in  the  detailed  sensitivity  analysis  for  real 
application. 

It  is  possible  that  the  system  architecture  threshold  requirements  are  based  on 
existing  systems,  and  because  these  systems  would  have  little  to  no  development  cost,  a 
Threshold  Only  solution  would  be  one  that  has  significantly  lower  cost  than  the  other 
system  architectures.  In  order  to  be  the  most  preferred  solution  the  maximum  cost  of  the 
Threshold  Only  system  is  $0.9B,  or  73%  lower  than  the  cost  of  System  Concept  2. 

The  Threshold  Only  system  is  not  competitive  with  eight  of  the  nine  system 
concepts.  This  is  despite  having  a  cost  that  is  already  40%  lower  than  the  next  least 
expensive  concept.  To  become  the  most  preferred  concept  the  cost  would  have  to 
decrease  an  additional  $1.1  (55%),  down  to  $0.9B,  or  the  OMOE  score  would  have  to  be 
increased  to  0.37  (206%).  It  must  be  noted  that  an  OMOE  score  of  0.38  would  exceed 
the  score  achieved  by  System  Concept  2  which  had  perfonnance  above  threshold  in  10  of 
the  1 1  key  perfonnance  attributes. 
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5. 


Attribute  Weighting  Effect 


To  check  the  effect  of  the  selected  attribute  weighting  an  analysis  was  run  with  all 
attributes  weighted  equally  at  1/11.  The  results  are  shown  in  Table  15,  with  the  most 
preferred  concept  remaining  unchanged  (System  Concept  2).  The  cost  per  OMOE  per 
year  at  FOC  for  System  Concept  1  increased  significantly  from  0.61  to  0.78  (27%)  and 
fell  in  preference  to  3rd  place,  indicating  the  benefit  gained  with  the  emphasis  on 
Capacity  and  the  year  FOC  is  achieved  through  their  increased  weighting. 


Table  15.  Cost  per  OMOE  per  year  at  FOC  Calculation  and  Ranking  Results  with  Equally 

Weighted  Performance  Attributes. 


System  Architecture 

Cost  ($B) 

Years  at  FOC 

OMOE  Score 

Cost/OMOE/FOCyear 

Rank 

System  Concept  1 

3.6 

10 

0.53 

0.78 

3 

System  Concept  2 

3.3 

10 

0.65 

0.50 

1 

System  Concept  3 

4.5 

10 

0.40 

1.28 

9 

System  Concept  4 

6.0 

10 

0.74 

0.80 

4 

System  Concept  5 

5.7 

10 

0.58 

1.08 

7 

System  Concept  6 

3.8 

10 

0.46 

0.94 

5 

System  Concept  7 

5.1 

10 

0.45 

1.36 

10 

System  Concept  8 

4.7 

10 

0.51 

1.07 

6 

System  Concept  9 

4.7 

10 

0.69 

0.76 

2 

Threshold  Only 

2 

10 

0.20 

1.14 

8 

6.  Variation  of  the  FOC  Period 

The  time  at  FOC  for  the  model  has  been  set  to  a  constant  of  10  years.  To  analyze 
the  effect  of  making  varying  this  input  new  values  for  all  concepts  have  been  replaced 
with  a  random  value  between  8  and  11.  The  results  are  shown  in  Table  16.  While  there 
are  some  changes  in  ranking,  System  Concept  2  remains  the  most  preferred,  even  with 
the  increased  cost  per  OMOE  per  year  at  FOC. 

7.  Range  of  Variation  of  the  Cost 

It  may  be  argued  that  the  cost  variation  in  the  example  at  a  factor  of  2  is  too  large 
and  biasing  the  results.  To  address  this  concern  the  existing  cost  variations  were  mapped 

from  the  $3-6B  range  over  to  a  $3-4B  range.  This  would  maintain  the  cost  comparisons 
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between  system  concepts,  but  decrease  the  total  input  cost,  and  as  such  the  ratio  of  cost  to 
OMOE.  The  results  are  shown  in  Table  17. 

This  change  in  range  does  have  an  effect  on  the  ranking  of  the  third  through 
seventh  place  concepts,  it  is  still  not  sufficient  to  change  the  results  for  the  two  most 
preferred  solutions.  This  result  shows  that  the  smaller  the  range  of  variation  is  on  the 
cost,  the  more  meaningful  the  variation  of  the  OMOE  will  become  in  the  decision  making 
process. 


Table  16.  Revised  Ranking  With  a  Variable  FOC  Period. 


System  Architecture 

Cost  ($B) 

Years  at  FOC 

OMOE  Score 

Cost/OMOE/FOCyear 

Rank 

System  Concept  1 

3.6 

8 

0.59 

0.76 

3 

System  Concept  2 

3.3 

8 

0.60 

0.69 

1 

System  Concept  3 

4.5 

9 

0.35 

1.44 

9 

System  Concept  4 

6.0 

10 

0.71 

0.85 

4 

System  Concept  5 

5.7 

11 

0.57 

0.91 

6 

System  Concept  6 

3.8 

10 

0.42 

0.91 

5 

System  Concept  7 

5.1 

10 

0.44 

1.15 

8 

System  Concept  8 

4.7 

9 

0.48 

1.10 

7 

System  Concept  9 

4.7 

10 

0.62 

0.76 

2  ' 

Threshold  Only 

3.9 

10 

0.17 

2.25 

10 

Table  17.  Revised  Ranking  With  a  Reduced  Cost  Variation  Range. 


System  Architecture 

Cost  ($B) 

Years  at  FOC 

OMOE  Score 

Cost/OMOE/FOCyear 

Rank 

System  Concept  1 

3.2 

10 

0.59 

0.54 

2 

System  Concept  2 

3.1 

10 

0.60 

0.52 

1 

System  Concept  3 

3.5 

10 

0.35 

1.01 

9 

System  Concept  4 

4.0 

10 

0.71 

0.56 

3 

System  Concept  5 

3.9 

10 

0.57 

0.69 

5 

System  Concept  6 

3.3 

10 

0.42 

0.78 

7 

System  Concept  7 

3.7 

10 

0.44 

0.83 

8 

System  Concept  8 

3.6 

10 

0.48 

0.75 

6 

System  Concept  9 

3.6 

10 

0.62 

0.58 

4 

Threshold  Only 

3.3 

10 

0.17 

1.91 

10 

34 


E. 


CHAPTER  SUMMARY 


This  chapter  has  discussed  the  application  of  the  mathematical  model  in  an 
example  communications  satellite  system  architecture  trade  study.  A  set  of  nine  different 
system  architectures  and  eleven  key  attributes  were  investigated.  The  model  was 
implemented  in  an  Excel  workbook.  Perfonnance  values  were  created  by  a  random 
number  generator,  and  the  calculation  results  were  presented  and  discussed.  A 
comparison  of  the  mathematical  model  results  to  the  traditional  cost  per  capacity 
calculation  shows  that  there  can  be  different  recommended  solutions  for  the  two 
approaches.  This  difference  in  results  shows  that  if  other  key  system  attributes  are 
ignored  it  is  likely  that  a  system  architecture  solution  will  be  selected  that  is  not 
optimized  for  either  cost  or  perfonnance. 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


This  chapter  presents  a  summary  of  the  work  described  in  this  thesis  and 
identifies  areas  for  further  research. 

A.  CONCLUSIONS  AND  RECOMMENDATIONS 

The  purpose  of  this  thesis  was  to  implement  a  mathematical  model  for  assessing 
communications  satellite  system  architectures  based  on  the  satisfaction  of  multiple 
perfonnance  attributes.  To  this  end  the  following  three  questions  have  been  addressed: 

1.  What  are  the  key  quantifiable  architectural  attributes  that  contribute  to 
meeting  the  users’  requirements  of  a  communications  satellite? 

2.  What  is  an  appropriate  mathematical  model  for  evaluating  communications 
satellite  architectures? 

3.  How  can  such  a  mathematical  model  be  applied  to  the  assessment  of  a 
communications  satellite  architecture? 

Satellite  based  communications  provide  a  crucial  capability  to  the  U.S.  military  in 
the  execution  of  their  many  missions.  System  architectures  are  developed  based  on  a  set 
of  requirements  that  define  the  capabilities  needs.  Traditionally  the  selection  of  a 
communications  satellite  systems  architecture  solution  is  based  upon  the  evaluation  of  a 
single  criterion:  the  cost  per  unit  of  system  communication  capacity.  This  selection 
approach  disregards  the  other  key  system  design  requirements  and  can  result  in  the 
selection  of  an  architectural  solution  that  does  not  reflect  other  system  attributes  the 
stakeholders  may  consider  significant.  To  address  this  shortcoming,  a  mathematical 
model  to  assess  multiple  attributes  simultaneously  was  implemented. 

A  survey  of  candidate  system  parameters  for  inclusion  into  a  mathematical  model 
for  trade  study  analysis  of  candidate  communications  satellite  architectures  was 
conducted.  The  five  identified  potential  key  attributes  are:  communications  capacity; 
access;  interoperability;  commandability;  and  information  assurance  and  protection.  Due 
to  the  commonality  with  current  trade  analyses  and  commercial  leasing  metrics,  the 
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communications  capacity  attribute  should  be  considered  applicable  to  all  system 
architecture  trade  analyses.  Additional  key  attributes  should  be  utilized  in  the 
mathematical  model  based  on  the  defined  requirements  for  the  system,  such  as  KPPs, 
TPMs,  and  overarching  requirements. 

Three  mathematical  model  approaches  were  discussed.  The  selected  approach  is  a 
combination  of  two  of  them  and  enables  a  comparison  of  the  cost  for  achieved 
perfonnance  level.  This  approach  leveraged  three  mathematical  model  benefits:  1)  the 
ability  for  stakeholders  to  weight  the  relative  importance  of  all  key  attributes  utilized  in 
the  model;  2)  a  calculation  of  raw  scores  to  measure  the  performance  of  each  key 
attribute  across  system  architectures;  and  3)  consideration  of  the  total  system  cost  for 
each  system  architecture. 

The  inputs,  output,  and  calculation  methodology  of  the  selected  mathematical 
model  were  presented.  The  output  of  the  mathematical  model  is  a  cost  per  year  at  FOC 
per  Overall  Measure  of  Effectiveness  (OMOE)  score.  The  lower  the  cost  per  OMOE 
score,  the  more  preferred  the  system  architecture  is  relative  to  the  others  under 
consideration.  This  results  in  a  metric  for  decision  makers  to  utilize  in  their  system 
architecture  selection  where  the  annualized  cost  per  unit  of  perfonnance  is  minimized. 

The  mathematical  model  was  applied  to  an  example  communications  satellite 
system  architecture  trade  study.  A  set  of  nine  different  system  architectures  and  eleven 
key  attributes  were  investigated.  The  model  was  implemented  in  an  Excel  workbook  and 
performance  values  were  created  by  a  random  number  generator  within  the  bounds  of 
expected  values  for  a  fictitious  system.  The  output  of  the  mathematical  model  showed 
the  lowest  cost  per  capability  can  be  lower  than  the  cost  per  capacity.  The  difference  in 
results  supports  applying  the  crucial  step  of  accounting  for  additional  key  system 
attributes  when  selecting  a  communications  satellite  system  architecture. 

This  model  is  intended  to  aid  decision  makers  in  a  system  architecture  selection 
process.  Decision  makers  will  still  have  to  consider  the  results  of  the  model  in  concert 
with  other  factors  that  do  not  lend  themselves  to  incorporation  into  a  mathematical 
model,  such  as  budget  constraints,  rapidly  evolving  threats  and  needs,  and  overall  level  of 
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program  risk.  An  analysis  of  the  results  will  be  necessary  to  support  the  final  decision, 
with  the  approach  and  focus  to  be  defined  by  the  needs  of  the  decision  makers,  ft  is  also 
possible  that  the  results  of  this  approach  may  yield  a  result  that  is  not  different  from  that 
produced  by  the  traditional  cost  per  capacity  approach.  In  this  case,  this  approach  would 
be  a  validation  of  the  traditional  approach. 

B.  AREAS  FOR  FURTHER  RESEARCH 

There  are  a  number  of  areas  where  this  research  could  be  expanded  in  further 
research  as  a  result  of  this  effort.  Three  possibilities  are  suggested. 

The  mathematical  model  could  be  applied  to  existing  or  proposed  system 
architecture  to  detennine  if  there  is  any  difference  between  the  system  selected  by  the 
traditional  cost  per  capacity  method  and  this  multi-attribute  mathematical  model.  This 
effort  would  include  identification  of  the  key  attributes,  determination  of  appropriate 
weightings  based  on  inputs  from  stakeholders,  execution  of  the  mathematical  model,  and 
an  analysis  of  the  results.  This  research  could  help  to  demonstrate  the  utility  of  the 
mathematical  modeling  approach. 

Another  potential  research  area  would  be  the  expansion  of  the  model  to 
accommodate  more  complex  types  of  functionalities  between  the  satellites  in  the  system 
architecture  and  the  user  tenninals.  This  would  enable  the  modeling  of  a  more  complex 
solution  that  includes  a  variety  of  different  elements  assembled  to  create  an  integrated 
system  architecture  solution.  Examples  of  such  an  architectural  solution  include: 
satellites  of  differing  capabilities  and  in  different  orbits  to  provide  full  earth  coverage;  a 
mix  of  military  satellites  and  commercially  leased  capacity;  and  the  inclusion  of 
atmospheric  assets  such  as  balloons  and  aircraft. 

The  mathematical  model  can  be  expanded  to  handle  time-varying  values,  such  as 
the  time  value  of  money,  or  even  statistically  varying  inputs.  This  type  of  analysis  could 
provide  an  even  more  relevant  outcome  for  use  by  the  stakeholders. 
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